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(57) ABSTRACT 

A system and method for dynamically modifying parameters 
in a data storage system such as a RAID system. Such 
parameters include QOS (Quality of Service) parameters, 
which control the speed in which system operations are 
performed for various parts of a data storage system. The 
storage devices addressable as logical volumes can be 
individually controlled and configured for preferred levels of 
performance and service. The parameters can be changed at 
any time while the data storage system is in use, with the 
changes taking effect very quickly. These parameter changes 
are permanently stored and therefore allow system configu- 
rations to be maintained. A user interface (Uf) allows a user 
or system administrator to easily observe and configure 
system parameters, preferably using a graphic user interface 
which allows a user to select system changes along a scale 
from minimum to a maximum. 

18 Claims, 3 Drawing Sheets 
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DYNAMICALLY MODIFYING SYSTEM Such parameters include QOS (Quality of Service) 

PARAMETERS IN DATA STORAGE SYSTEM parameters, which control the speed and functionality in 

which system operations are performed for various parts of 
a data storage system. 

HELD OF THE INVENTION 5 According to the invention, a user interface is provided 

This invention is directed towards mass data storage for the data storage system to allow control over certain 

systems for computers, and more particularly towards provided services on a logical volume basis. These services 

manipulation of storage system functional parameters. generally fall into three major categories: data replication/ 

recreation through a "copy" mechanism, performance man- 

BACKGROUND io agement through control of caching services, and data 

Mass data storage systems have become prevalent in integrity checks, 

information systems due to the need for flexibility, high The data replication/recreation function can be broken 

availability, and extremely large storage capacities. Such down into 4 sub-categories: Business Continuance Volumes 

mass data storage systems typically include a plurality of (BCVs), Remote Data Facility (RDF), Secondary Storage 

disks arranged in a RAID (Redundant Array of Independent 15 Backup, and Service/Rebuild. Each category has its own 

Disks) or other configuration, and a memory cache to level of service that can be controlled with the API. In one 

increase data reading and writing speed. embodiment, the level of service is set in terms of a level 

Such data storage systems also typically provide sophis- from 0 t0 10 with 0 being the best level of service (also the 

ticated storage management, wherein features such as opti- default level of service) and 10 being the lowest level of 

mal cache management, data protection and backup proce- service. 

dures are automatically handled by the mass storage system. In an illustrative embodiment, the performance manage - 
Such features enable system users and administrators to ment function is performed by a user through a Graphic User 
utilize the mass storage systems with minimal concern about Interface (GUI) coupled to the data storage system that 
day-to-day optimization details. allows performance values to be selectable, for example, 
Although known mass data storage systems provide 25 from an ^creasing or decreasing range of values (i.e. 0-10 
sophisticated storage management, some customers and or 0-100). The GUI translates these values into an appro- 
users have a need to control some of the storage system P riate form > such as bitmasks, to select/deselect various 
behavior to ensure the quality of service (QOS). For mass storage system services, and sends the bitmasks 
example, it would be beneficial to identify certain volumes 30 trough an application program interface (API) to the data 
(logical, virtual, or physical disks) as having lower priority storage system. 

than others, or to modify levels of protection on individual The data integrity check category includes physical drive 

volumes. These behaviors are typically established by stored scrubbing services. These services detract from system 

system parameters which are loaded when the system is performance, but minimize the risk of head crashes, 

booted. System wide parameters, sometimes called Algo- 35 These quality of service functions are controlled through 

rithm Parameters, provide control of some of the caching the user interface. Once set, these functions remain set and 

services (and some other internal algorithm services) such as preserved across system resets/power downs until specifi- 

least recently used (LRU) placement for returned data, cally changed. The user interface can be used to change 

second chance decisions (retries), statistical LRU updates, these functions while the data storage system is running, 

etc. 40 Advantages of the present invention include the ability of 

In known data storage systems, these controls are done at users to define different performance requirements for dif- 

configuration time and are not changeable during normal ferent logical volumes in a data storage system. Logical 

operation. To change the service behavior, the system must volumes which are rarely accessed, or of minimal impor- 

be restarted to have the new parameters take effect. tance can have their QOS (Quality of Service) decreased, 

However, shutting down and restarting a high availability 45 thereby freeing up other system resources, including the 

data storage system is a complex task. The system must be cache footprint, CPU time, cycles, and hardware resources 

taken off-line, thereby making the stored data unavailable to such as bus and memory usage. This typically results in a 

the host computers, resulting in an interruption in service. In corresponding increase in the QOS of important logical 

mission-critical applications, even a short interruption in volumes. 

service may cause serious problems, including lost business 50 Other advantages of the present invention include the 

and financial opportunities. After the mass storage system is ability to make changes dynamically to the data storage 

offline, the cached data must all be written out to the disks, system while it is in use. The changes made to the QOS of 

and the system prepared to halt and restart. Once restarted, various system parameters take effect quickly enough to be 

the system may take several minutes to properly configure almost transparent to the system administrator, and to end 

(based on the changed parameters) and return to full avail- 55 users. This allows system administrators to observe perfor- 

ability. mance changes in real time, and thereby optimize the system 

Therefore, changing the system behavior in known data through immediate feedback, 

storage systems is a difficult and time consuming process. Other advantages include a fail-safe and fault tolerant 

Further, it is difficult to attempt to "tweak'* or tune system method for adjusting parameters of a data storage system, 

performance through incremental changes while observing 60 The adjusted parameters are permanently stored so that the 

the performance improvements after each incremental system is stable between power-downs and during power 

change. Generally, users are not able to make changes to the interrupts. The adjusted parameters are maintained until they 

system behavior while the system is running. are altered by a system administrator. The user interface 

nummary provides users and system administrators with an intuitive 

65 Graphic User Interface (GUI) for monitoring and changing 

The present invention provides a system and method for data storage system parameters and QOS settings. Users can 

dynamically modifying parameters in a data storage system. easily manipulate the system settings using the GUI, includ- 
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ing a with features such as bar graphs and scales showing respective one of a number of channels 20a-d. Although the 

real-time system settings and performance. data storage devices Ma-b are shown in close proximity, 

they may actually geographically remote from each other, 

BRIEF DESCRIPTION OF THE DRAWINGS for example using a remote data facility, which maintains 

™, r , iL r jj* 17*1.5 mirrored data at different locations. 

The foregoing and other features and advantages of the 

present invention will be more fully understood from the ^ bus 18 * typically the backplane of a printed-circuit 

following detailed description of illustrative embodiments, card-cage or mam-frame in cabinet, and each of the host 

taken in conjunction with the accompanying drawings in adapters 15a-*> and device controllers \9a-b is constructed 

wmCQ: on a printed circuit board that is mounted in the card-cage or 

- . .i i j* c "it j i * 10 main-frame in the cabinet. For redundancy purposes, the bus 

FIG. 1 is a block diagram of an illustrative data storage it> , , . , n J f, r . . , 

& 18 may also be a pair of buses where a first bus is connected 

^ ' to a first set of host adapters 15 and a first set of device 

FIG. 2 is a block diagram showing data flow according to controllers 19 with a second bus connected to a second set 

an illustrative embodiment of the present invention; and of host adapters 15 anc j a ^cond set of device controllers 19. 

FIG. 3 is a flow chart of a method implemented by the is Both buses are connected to system memory 13 and can 

illustrative embodiment of FIG. 2. operate concurrently. 

hptatt en npcruTPTTHM Th e system memory 13 is constructed on a number of 

utliULtJJ utun-Kif iiuw additional printed circuit boards that are mounted in the 

FIG. 1 shows a block diagram of an illustrative data card-cage or main-frame. The system memory includes a 

storage system 10 for implementing dynamic system param- 20 plurality of memory boards coupled to the bus or buses 18. 

eter modifications according to the present invention. The While multiple memory boards can support concurrent 

data storage system 10 includes a number of data storage operations, only one memory board can be accessed at one 

devices 11a, lib and a cache memory 12. Typically the data time by a host adapter or device controller. The system 

storage devices 11a, 116 are disk storage devices, each of memory 13 may also include a region of memory known as 

which may include one or more disk drives, dependent upon 25 permacache memory. As is well known, data elements 

the user's requirements and system configuration. However, remain in permacache memory unless they are specifically 

the data storage system 10 may use other kinds of storage deleted. 

devices, including limited to optical disks, CD ROMS and Before accessing the system memory 13, each host 

magnetic tape devices. 30 adapter ISa-b and device controller 19a-b must obtain 

The cache memory 12 is part of a system memory 13. In access to one of the buses 18 and the shared system memory 
the illustrative data storage system 10 the system memory 13 13. Moreover, to ensure that all of the processors have an 
is a high-speed random- access semiconductor memory. The equal opportunity to access the shared system memory, in 
system memory 13 also includes a cache index directory 14 the illustrative embodiment access is granted to another 
which provides an indication of the data which is stored in 35 processor in a round-robin fashion in the event that more 
the cache memory 12 and provides the addresses of the data than one of the other processors is seeking access at the time 
which is stored in the cache memory 12. The cache index that a processor relinquishes its access. This arbitration of 
directory is organized as a hierarchy of tables for devices, access requests is performed by conventional bus access 
which may be physical data storage devices, or logical logic 26 that receives access request signals from the pro- 
volumes. A logical volume, (LV) is a set of cylinders which 40 cessors linked to the bus 18, and provides grant signals to the 
form a virtual or physical disk. The data storage system 10 processors. The bus access logic 26 asserts a grant signal to 
can maintain the physical storage as a set of logical volumes only one processor at any given time to grant access to the 
for simplicity. The cache index directory maintains tables of bus 18. However, a wide variety of arbitration schemes are 
logical volume cylinders and tracks of data records, as known in the art that are suitable for use in a data storage 
further described in Yanai et al., U.S. Pat. No. 5,206,939, 45 system employing multiple processors and a shared system 
issued Apr. 27, 1993, and incorporated herein by reference. memory. 

The availability of access to the data in the data storage To provide reliable access by any host 16a-b over any 

system 10 is enhanced by employing multiple processors host channel 17a- d to any of the data stored in the data 

15a, 15b in the system for interfacing the system to one or storage devices lla-b, it is necessary to coordinate each of 

more host computers 16a, 16b through a plurality of data 50 the host adapters 15a— fc, with each of the device controllers 

access channels lla-d. Each of these processors 15a, 156 19a-6. To simplify this coordination, it is desirable to use 

that function as a host adapter, for example, provides an the system memory 13 and in particular the cache memory 

interface to a separate and distinct group of the host channels 12 as a buffer for data transfer between each host adapter and 

170-0*. For enhanced reliability, however, each host typi- each device controller. Such a system, for example, is 

cally is provided with a data access channel to more than one 55 described in Yanai et al., U.S. Pat. No. 5,206,939, issued 

of the host adapters. The host adapters ISa-b can access the Apr. 27, 1993, and incorporated herein by reference. In such 

system memory 13 through a high-speed, parallel-line sys- a system, it is not necessary to provide a processor dedicated 

tern bus 18, to managing the cache memory 12. Instead, each of the host 

To enhance data access availability to the data in the adapters or device controllers executes a respective cache 

storage devices Ma-b, additional processors 19a-b are 60 manager program 21o-b, 22a~b. Each of the cache manager 

typically used for controlling data access to the storage programs access the cache memory 12, cache index direc- 

devices. Each of these additional processors 19a-b that to ry 14, and cache manager memory 24. The cache manager 

functions as a device controller, for example, controls data memory 24 includes a queue 25 and a data structure 35 for 

access to a group of the data storage devices 11a-*. For storing pending write operations. 

enhanced accessibility and reliability, each data storage 65 The cache manager programs 21a-b of the host adapters 

device lla-b has two ports, each of which is interfaced to ISa-b write data from the hosts 16a-b into the cache 

a different one of the device controllers 19a-b over a memory 12 and update the cache index directory 14 and the 
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cache manager memory 24. The cache manager programs The present invention provides a system and method for 

2la-b of the host adapters ISch-b also access the cache index dynamically modifying parameters in a data storage system 

directory 14 and read data from the cache memory 12 for 10. Such parameters include Q OS parameters, which control 

transmission to the hosts 16a-b. system behavior on an individual or group of logical vol- 

The cache manager programs 22a-b of the device con- 5 umes. 

trailers 19a-b stage data from the storage devices Ha-fc to The present invention provides for an API for the data 

the cache memory 12 and update the cache index directory storage system 10 to allow control over some of the provided 

14 and the cache manager memory 24. The cache manager services on a logical volume basis. These services generally 

programs 22a-fc of the device controllers 19a-b also fall into three major categories: data replication/recreation 

de-stage or write-back data from the cache memory 12 to the 10 through a "copy" mechanism, performance management 

storage devices 11a— fr, and update the cache index directory through control of system services, and data integrity 

14, checks. 

Many different techniques are available for controlling The data replication/recreation function in the illustrative 

how the cache manager programs 22a~b utilize the cache embodiment can be broken down into 4 sub-categories: 

memory 12 in a most efficient manner for system through- 15 Business Continuance Volumes (BCVs), Remote Data Facil- 

put. Some such techniques are described in U.S. Pat. No. ity (RDF), Secondary Storage Backup, and Service/Rebuild. 

5,537,568, issued Jul. 7, 1996; U.S. Pat. No. 5,381,539, E«* category has its own level of service that can be 

issued Jan. 10, 1995; U.S. Pat, No. 5,682,500, issued Oct. controlled by the API. In the illustrative embodiment, the 

28, 1997; U.S. Pat. No. 5,706,467, issued Jan. 6, 1998; and l evel of service is set in terms of a level from 0 to 10 with 

U.S. Pat. No. 5,787,473, issued Jul. 28, 1998; which are all 20 0 being the best level of service (also the default level of 

incorporated herein by reference. service) and 10 being the lowest level of service. 

The data storage system 10 further includes a local or Performance management will be discussed in below. The 
remote system manager console 23 including an additional data integrity check category includes physical drive scrub- 
processor on an additional printed circuit board. The system bin S services. These services detract from system 
manager console 23 may be very remote, for example across 25 performance, but minimize the risk of head crashes. These 
the country from the actual data storage system 10, and services are also discussed below. 

connected by an appropriate telecommunications or network According to the present invention, a QOS data structure 

link. The system manager console 23 permits a system 42 FIG. 2 holds a plurality of QOS parameters, which may 

operator to run set-up and diagnostic programs for control be changed dynamically while the data storage system 10 is 

and monitoring of the performance of the data storage 30 running, and which will alter the QOS. A QOS data structure 

system 10. According to one feature of the present invention, is created for each storage device 11 in the data storage 

the system manager console 23 runs a User Interface (UI) 40 system 10. 

application which allows a user to modify the QOS (Quality In an illustrative embodiment of the present invention, 

of Service) for the system 10. The UI 40 displays data for the each device header (a data structure in system memory 13 

data storage system 10 in a user- friendly interface format, 35 that is associated with a logical volume, also known as 

and allows a user to monitor and change performance logical volume parameters) includes a QOS data structure 

parameters. In an illustrative embodiment, a graphic user 42, either directly or through a link. The QOS data structure 

interface (GUI) allows performance parameters to be 42 is created with the following fields initially set to zero, as 

selected from a range of values (i.e. 0-10 or 0-100) or with shown in Table 1: 



TABLE 1 






QOS Data Structure 




Field Name 


Field Size 


Range 


Description 


BCV Copy Priority 


Byte 


0-10 


Priority level for BCV 








(business continuance volumes) 








Copies 


RDF Copy Priority 


Byte 


0-10 


Priority level for RDF (remote 








data facility) Copies 


Secondary Storage 


Byte 


0-10 


Priority level for Secondary 


Backup Copy Priority 






Storage Backup Copies 


Service Copy Priority 


Byte 


0-10 


Priority level for 








Service/Maintenance Copies 


Disk Scrub Priority 


Byte 


0-10 


Priority level for physical disk 








scrubbing 


TBD 


3 Bytes 


TBD 


Reserved 


Cache Services 


32 bits 


0-0xFFFFFFFF 


Cache services selection bitmap 








(see Table 2) 


TBD 


4 Bytes 


TBD 


Reserved 



a sliding bar or scale. The GUI translates the parameter 
ranges into an appropriate form to select/deselect or modify 
various mass storage system services, ands uses an API to 
transmit the information to the system 10. 

An illustrative data storage system 10 such as described, 
is the Symmetrix® line of RAID -based data storage systems 
available from EMC Corporation of Hopkinton, Mass. 



Any changes to the corresponding QOS values are sub- 
sequently stored in the appropriate fields in the QOS data 
structure. Any changes to the device headers causes the 
devices headers to be marked for destaging to device con- 
65 troller local memory 19, as shown by arrow 46, and to disk 
backup. The changed QOS data structures 42 are also copied 
and backed up. Destaging takes place at predetermined 
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intervals, such as every 60 seconds, wherein the device increase during a QOS copy can cause the copy time to be 

headers or logical volume parameters 42 are copied to disk exceeded. 

11. Once the device controller 19 obtains the changed Fourth, since copies are done one logical volume at a 
parameters, it reconfigures the QOS for the logical volume time, the actions of the logical volumes not currently copy- 
based on the changed parameters. This results in changed $ ing can have a large impact on the systems ability to meet 
QOS for that logical volume. the copy window. Again this is a matter of a guaranteed level 

If a power interruption occurs, the illustrative embodi- of service, but the problem is one of coordination between 

ment includes backup systems which include providing logical volumes residing on the same physical volume. Fifth, 

temporary power to allow the data storage system 10 to write despite the attempt to limit the spike effect at the end of the 

out all pending data from the cache and perform an orderly 1Q QOS copy period, the spike effect remains due to the attempt 

shutdown. Therefore, no data (including logical volume to meet the QOS Copy period requirements. Finally, QOS 

parameters 42) is lost. If there is an interruption, the system, mode intentionally extends the time required to validate all 

upon power-up restores the logical volume parameters 42 tracks, but it unintentionally does this in an unfair way. The 

and commences operating in the proper configuration. first logical volume on the physical volume will have its time 

Preliminary techniques for modifying QOS related algo- to complete extended less than latter logical volumes on the 

rithms were implemented to provide an interim solution to 15 physical volume. So some volumes are more exposed than 

QOS needs. For example with respect to QOS mode, it was others are. 

previously possible at a system wide level to setup a QOS The present invention resolves these issues and make this 
mode for data replication/recreation, regardless of the copy QOS mode more useful in the general case. Unlike the 
type. Copying is performed when a physical or logical previous QOS implementation, this design avoids trying to 
volume is the be replaced, and the data copied to the new 20 detect the "start" of copy operations, since for many copy 
device. In this mode, the time for a device (such as a physical types the invalidation of tracks is an ongoing process, 
disk) to be copied was selected in minutes (0-65535). Any Instead of setting the copy period, which is difficult to define 
time that a QOS Start signal was detected the system would and tune, the user selects the level of service for the copies, 
attempt to schedule copies such that they would complete According to the illustrative embodiment, for each of the 
within the chosen time. 25 four copy operation types (RDF, BCV, Secondary Storage 
In order to detect QOS starts (with no clear hook for the Backup, and Service) the priority level sets a fixed inter- 
many different copy modes) the system watches the total track copy delay. The formula for converting priority level 
invalid track counts for each physical disk (a sum of the to inter-track copy delay is as follows: 
invalid tracks for each logical volume on the physical) on a ^tnck copy delay (in mmiseconds^riority level) 3 (1) 
periodic basis (typically 30 seconds). If this count grew by 30 

more than a fixed amount (typically 1000 tracks) from the ^ formula allows for very large delays (on the order 

prior period, then a QOS start was detected. seconds) between tracks, and for minimal delays (on the 

For RDF (remote data facility) devices, the QOS start had order of milliseconds), 

an additional restriction that starts would not be detected In order t0 provide preferential treatment of the first 

(and periodic invalid counts not saved) while the RDF links 35 logical volumes, a forced interleaving of copies from all the 

were disabled. This allowed detection of a QOS start when different logical volumes on the physical record is per- 

the invalid track counts are increasing at a slow rate (due to formed. This is performed by moving the 'copy data struc- 

writes) while the links are down. ture ' to me logical device record from the physical device 

Once the QOS start was detected, an inter-track copy time rccord ( a tolal of onl y 3 b y tes > when interleaving, it is 

was calculated by dividing the requested copy time by the 40 wa ^ orimi to kce P thc actuator focus long enough to avoid 

number of tracks that needed to be copied. This time was wastmg unnecessary time in actuator movement, 

then used as the delay between track copies. The delay was In the illustrative embodiment, the forced interleaving 

recalculated periodically (every 30 seconds) to assure accu- ^ that eacn lo S ical volu nie is given is calculated as 

racy. follows: 

However, due to some reasons discussed below, it is 45 Interleave time (in miUiseconds)-N* (average track copy execution 

possible for the QOS copies time to expire before all the time)+max (biter-track copy times of all LV's on physical), (2) 

tracks have been copied. When this happens a minimum The average track copy execution time is estimated at 40 

inter-track copy delay is set until all tracks are copied. This ms. N is the target number of tracks to be copied per interval 

limits the "spike effect" seen at the end of a QOS copy period. Typically, N is set to be 1000. This creates a range 

operation. 50 with a maximum interval of (1000 ms+40 ms)* 1000=1 040 

This QOS mode has a few inherent problems. First, a seconds (17+mmutes) and a minimum period of 40 seconds, 

uniform copy rate is assumed for all logical volumes, but in After the expiration of the interleave period, the next 

reality the copy rate depends greatly on the level of protec- logical volume on the physical disk will be selected for 

tion each logical volume has, its placement on the physical round robin scheduling. 

volume, etc. For example, a physical volume with a mix of 55 The illustrative embodiment of the present invention 

RAID-S (striping), Mirror, and Single disk hypers (a single allows for a large variety of QOS parameters to be set for 

mirror having a host addressable unit). This issue can cause each logical volume. The API (application program 

the requested copy time to be exceeded. Second, after interface) 23 FIG. 2 supports a 32-bit bitmap per logical 

expiration of the inter-track copy time, the copy operation is volume that allows selection/deselection of a range of cache 

not guaranteed, and can be delayed by other I/O type 60 services. The external API 23 and User Interface 40 display 

operations such as reads and writes to the same logical and convert a user-friendly level of service (for example a 

volume. This is partially handled by the readjustment of the sliding scale from 0-10 or 0-100) into an appropriate bit 

inter-track copy times every 30 seconds, but does not mask. This provides the maximum flexibility without micro- 

necessarily solve the problem. Third, the implementation code changes to the data storage system 10. 

assumes a one-time event where many tracks are marked as 65 The initial cache services in the illustrative embodiment 

invalid, and then there is a period where the invalid tracks that can be controlled are shown in Table 2. Bit 0 is the least 

can be copied. Events that cause the invalid track count to significant bit (Big Endian format). 
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TABLE 2 



Controllable Cache Services 
Bit Number Description/Effect 



0 Inhibit statistical LRU (least recently used) updates. Mates the LRU 
work as a FIFO LRU-saves overhead in LRU updates, but can reduce 
the hit rate for the LV 

1 Hie prefetch algorithm should always perform a Cut regardless of the 
average LRU fall through time. Prevents cache pollution due to 
sequential streams on systems with long LRU fall through times, but can 
reduce the hit rate for the LV 

2 Don't give cache slots a second chance on the LRU queue. Saves extra 
updates of the LRU, but can reduce the LV's hit rate. 

3 Disable Dynamic Max Write Pending feature for LV. Limits maximum 
number of write pending slots for the logical volume which can improve 
read response on other LV*s, can increase disco nnect time for the LV. 

4 Return slots that have been written to the bottom of the LRU instead of 
using a dynamic algorithm to determine slot placement. Can improve 
read hit rate of volumes, but with the possible expense to current LV of 
read hits on data that has been written. 

5-31 Reserved 



The steps performed by the illustrative embodiment of the 
present invention are shown in FIG. 3. The data storage 
system reads the stored QOS parameters, which either 
occurs at the update interval or upon system configuration, 
step 100. The data storage system then configures the QOS 
settings appropriately for the logical volume, step 102. 

To allow system administrators to view and adjust the 
system QOS parameters, the system also accesses and reads 
the stored QOS parameters, step 104, to be displayed by the 
user console through the UI (user interface). Typically the 
QOS parameters are read out of the system memory. The UI 
then displays the current QOS settings using a graphic user 
interface (GUI) to the system administrator, step 106. Any 
time the system administrator wishes to change the 
parameters, they may easily do so using the graphic user 
interface, step 108. The UI then sends the changes to the data 
storage system 10 to allow the QOS parameters to be 
updated, step 110. If the system administrator attempts to set 
any parameters with values that are not valid (for example, 
parameter settings that are contradictory when taken 
together, or create a tautology), the data storage system 10 
signals an error to the UI, which then displays an appropriate 
message to the system administrator. 

In an illustrative embodiment based on the Symmetrix 
data storage system, the access and modification of the 
Symmetrix Quality of Services feature set is performed 
through the Symmetrix system call interface. To read the 
current QOS values, as shown in step 104, the API uses the 
system function "Read Current QOS Values" with param- 
eters (qosvalues, 0168). This function is a passive system 
call that returns the current QOS values for a given range of 
logical volumes. The first 16 bytes are the standard prefix 
that is always returned. Table 3 shows QOS value records for 
the following bytes: 

TABLE 3 



QOS 'Values Prefix Format 

Bytes Count Format Meaning 

0000-0001 2 Word Number of valid records 

0002-0003 2 Word Size of each record in bytes 



25 

TABLE 3-continued 



QOS Values Prefix Format 
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Bytes 


Count 


Format Meaning 


0004-0005 


2 


Word First record number 


0006-0007 


2 


Word Max number of records 


0008-000f 


8 


Byte Reserved 


Every device record is in the format shown in Table 4: 






TABLE 4 




QOS Values Record Format 


Bytes Count 


Format 


Meaning 


0000-0000 1 


Byte 


Priority level for BCV Copies 


0001-0003 1 


Byte 


Priority level for RDF Copies 


0002-0002 1 


Byte 


Priority level for Secondary Storage 






Backup Copies 


0003-0003 1 


Byte 


Priority level for Service/Maintenance 






Copies 


0004-0004 1 


Byte 


Priority level for physical disk scrubbing 


0005-0007 3 


Byte 


Reserved 


0008-000b 4 


Long 


Cache services selection bitmap 




(see Thble 2) 


OOOc-OOOf 4 


Byte 


Reserved 



55 

To change the QOS values, step 110, the API uses the 
system function "Change Current QOS Values" with param- 
eters (qoscntrl, 813f). This is an active system call that 

6Q allows the changing of the current QOS values for a given 
logical volume. The extended parameters give control over 
the QOS values for a single device. Changes to QOS values 
are preserved across resets/power downs etc. To reset QOS 
values, a new change request must be made with all param- 

65 eters zero. QOS values are updated from global memory 
every 60 seconds. 

The format of these parameters is shown in Table 5: 
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QOS Control Record Format 



Bytes 


Count 


Format 


Meaning 


Range 


0000-0001 


2 


Word 


Logical Volume Number 


N/A 


0002-0003 


2 


Word 


Reserved 


N/A 


0004-0004 


1 


Byte 


Priority level for BCV Copies 


0-10 


0005-0005 


1 


Byte 


Priority level for RDF Copies 


0-10 


0006-0006 


1 


Byte 


Priority level for Secondary Storage Backup 


0-10 








Copies 




0007-0007 


1 


Byte 


Priority level for Service/Maintenance 


0-10 








Copies 




0008-0008 


1 


Byte 


Priority level for physical disk scrubbing 


0-10 


0009-OOOb 


3 


Byte 


Reserved 


N/A 


OOOc-OOOf 


4 


Long 


Cache services selection bitmap (see Table 2) 


0-Oxffffffff 


0010-0017 


S 


Byte 


Reserved 


N/A 
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Although the invention has been shown and described 
with respect to illustrative embodiments thereof, various 20 
other changes, omissions and additions in the form and 
detail thereof may be made therein without departing from 
the spirit and scope of the invention. 

What is claimed is: 

1. In a data storage system including a cache memory and 25 
a plurality of storage devices accessible as logical volumes, 

a method of dynamically changing system behavior on a 
logical volume basis, comprising: 

reading stored system parameters defining system behav- 
ior at least one logical volume; 30 

altering system behavior for said at least one logical 
volume to match said stored system parameters; 

facilitating user-requested changes to said system 
behavior, via a user interface application component 
configured to allow said user to monitor said system 
behavior and to allow said user to initiate real-time 
changes to said system behavior, by selecting a value 
along a range from a minimum to a maximum; 

receiving an indication of said user selected value of 4Q 
user-requested changes to system behavior for said at 
least one logical volume; 

mapping said user selected value to an appropriate value 
to modify said stored system parameters to match said 
user selected value for said system behavior by con- 45 
verting said user selected value into data to appropri- 
ately modify said system parameters stored in cache 
memory; 

dynamically altering system behavior for said at least one 
logical volume in accordance with said user selected 50 
value; and 

storing system parameters that define said altered system 
behavior for said at least one logical volume. 

2. The method of claim 1 wherein said user- requested 
changes are processed by a user interface application, said 55 
user interface application converting changes requested by 
said user into a format usable by said system to alter said 
system behavior. 

3. The method of claim 2 wherein said system behavior is 
defined by a set of parameters provided for each logical 60 
volume. 

4. The method of claim 1 wherein said system behavior 
includes cache performance. 

5. The method of claim 1 wherein said system behavior 
includes remote data file performance. 65 

6. The method of claim 1 wherein said system behavior 
includes copy performance. 



7. The method of claim 6 wherein modifying copy per- 
formance system behavior includes providing a forced inter- 
leaving of copies from all logical volumes. 

8. The method of claim 1 wherein said step of storing 
system parameters includes storing said system parameters 
on one of said plurality of storage devices, wherein said 
system parameters are stored in a fault tolerant manner. 

9. The method of claim 1 wherein said step of storing 
system parameters includes storing said system parameters 
in said cache memory, wherein subsequently said system 
parameters are copied to at least one of said plurality of 
storage devices. 

10. In a storage system including a cache memory and a 
plurality of storage devices accessible as logical volumes, a 
method of dynamically changing system behavior on a 
logical volume basis, comprising: 

reading stored system parameters defining system behav- 
ior for at least one logical volume; 

altering system behavior for said at least one logical 
volume to match said stored system parameters; 

receiving an indication of user-requested changes to sys- 
tem behavior for said at least one logical volume; 

dynamically altering system behavior for said at least one 
logical volume in accordance with said user-requested 
changes; and 

storing system parameters that define said altered system 

behavior for said at least one logical volume; 
wherein said system behavior includes copy performance; 
wherein modifying copy performance system behavior 

includes providing a forced interleaving of copies from 

all logical volumes; 
wherein a time value for said forced interleaving is 

calculated by: 

Interleave time«N* (average track copy execution time)+max 
(inter-track copy times of all LVs on physical volume); 

wherein N is a target number of storage device tracks to 
be copied per interval period. 

11. In a data storage system including a cache memory 
and a plurality of storage devices accessible as logical 
volumes, a method of dynamically changing system behav- 
ior on a logical volume basis, comprising: 

reading stored system parameters defining system behav- 
ior for at least one logical volume; 

altering system behavior for said at least one logical 
volume to match said stored system parameters; 

receiving an indication of user-requested changes to sys- 
tem behavior for said at least one logical volume; 
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dynamically altering system behavior for said at least one 
logical volume in accordance with said user-requested 
changes; and 

storing system parameters that define said altered system 
behavior for said at least one logical volume; 

wherein said system behavior is one of the set of copy 
priority, RDF copy priority, secondary storage backup 
copy priority, service copy priority, and disk scrub 
priority. 

12. A data storage system including a cache memory and 
a plurality of storage devices accessible as logical volumes, 
said system comprising: 

system parameters defining system behavior for at least 
one logical volume, said system parameters stored in 
cache memory, and also replicated on one of said 
plurality of storage devices; 

an interface component, to receive from a user indications 
of changed system parameters defining new system 
behavior for said at least one logical volume; 

a user interface application component, coupled to said 
interface component, to allow said user to monitor said 
system behavior, and to allow said user to initiate 
real-time changes to said system behavior, said user 
interface application component operable to convert 
user initiated real-time changes into data to appropri- 
ately modify said system parameters stored in cache 
memory, said user interface application component 
allowing said user to change said system behavior by 
selecting a value along a range from a minimum to a 
maximum and mapping said selected value to an appro- 
priate value to modify said system parameters to match 
said user selected value for said system behavior, 
whereupon said data storage system modifies said 
system behavior for said at least one logical volume; 
and 

said data storage system replicates said modified system 
parameters on one of said plurality of storage devices. 

13. The data storage system of claim 12 further including: 
a user interface application component, coupled to said 

interface component, to allow said user to monitor said 
system behavior, and to allow said user to initiate 
real-time changes to said system behavior. 

14. The data storage system of claim 13 wherein said user 
interface application includes a graphic user interface (GUI) 
that converts user initiated real-time changes into data to 
appropriately modify said system parameters stored in cache 
memory. 

15. The data storage system of claim 12 wherein said 
system behavior for at least one logical volume includes 
cache performance for said at least one logical volume. 

16. The data storage system of claim 15 wherein said 
system behavior for at least one logical volume includes 
remote data file performance for said at least one logical 
volume. 

17. A data storage system including a cache memory and 
a plurality of storage devices accessible as logical volumes, 
said system comprising: 



system parameters defining system behavior for at least 
one logical volume, said system parameters stored in 
cache memory, and also replicated on one of said 
plurality of storage devices; 
5 an interface component, to receive from a user indications 
of changed system parameters defining new system 
behavior for said at least one logical volume; 

wherein said interface component, upon receiving said 
10 indications of changed system parameters, appropri- 
ately modifies said system parameters stored in cache 
memory, whereupon said data storage system modifies 
said system behavior for said at least one logical 
volume; and 

15 said data storage system replicates said modified system 
parameters on one of said plurality of storage devices; 
and 

a user interface application component, coupled to said 
20 interface component, to allow said user to monitor said 
system behavior, and to allow said user to initiate 
real-time changes to said system behavior, wherein said 
user interface application includes a graphic user inter- 
face (GUI) that converts user initiated real-time 
25 changes into data to appropriately modify said system 
parameters stored in cache memory; 
wherein said GUI allows said user to change said system 
behavior by selecting a value along a range from a 
minimum to a maximum, and said GUI maps said 
30 selected value into to an appropriate value to modify 
said system parameters to match said user selected 
value for said system behavior. 
18. A data storage system including a cache memory and 
a plurality of storage devices accessible as logical volumes, 
said system comprising: 

system parameters defining system behavior for at least 
one logical volume, said system parameters stored in 
cache memory, and also replicated on one of said 
40 plurality of storage devices; 

an interface component, to receive from a user indications 
of changed system parameters defining new system 
behavior for said at least one logical volume; 
wherein said interface component, upon receiving said 
indications of changed system parameters, appropri- 
ately modifies said system parameters stored in cache 
memory, whereupon said data storage system modifies 
said system behavior for said at least one logical 
50 volume; and 

said data storage system replicates said modified system 
parameters on one of said plurality of storage devices; 
wherein said system behavior is one of the set of copy 
priority, RDF copy priority, secondary storage backup 
55 copy priority, service copy priority, and disk scrub 
priority. 
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